A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E).

BACKGROUND: Observational epidemiologic studies provide critical data for the evaluation of the potential effects of environmental, occupational and behavioural exposures on human health. Systematic reviews of these studies play a key role in informing policy and practice. Systematic reviews should incorporate assessments of the risk of bias in results of the included studies. OBJECTIVE: To develop a new tool, Risk Of Bias In Non-randomized Studies - of Exposures (ROBINS-E) to assess risk of bias in estimates from cohort studies of the causal effect of an exposure on an outcome. METHODS AND RESULTS: ROBINS-E was developed by a large group of researchers from diverse research and public health disciplines through a series of working groups, in-person meetings and pilot testing phases. The tool aims to assess the risk of bias in a specific result (exposure effect estimate) from an individual observational study that examines the effect of an exposure on an outcome. A series of preliminary considerations informs the core ROBINS-E assessment, including details of the result being assessed and the causal effect being estimated. The assessment addresses bias within seven domains, through a series of 'signalling questions'. Domain-level judgements about risk of bias are derived from the answers to these questions, then combined to produce an overall risk of bias judgement for the result, together with judgements about the direction of bias. CONCLUSION: ROBINS-E provides a standardized framework for examining potential biases in results from cohort studies. Future work will produce variants of the tool for other epidemiologic study designs (e.g. case-control studies). We believe that ROBINS-E represents an important development in the integration of exposure assessment, evidence synthesis and causal inference.

Integrated Risk Information System (IRIS) response to "Assessing risk of bias in human environmental epidemiology studies using three tools: different conclusions from different tools".

"Assessing risk of bias in human environmental epidemiology studies using three tools: different conclusions from different tools," a recent publication in this journal, applied the study evaluation approach developed by the U.S. Environmental Protection Agency's Integrated Risk Information System (IRIS), as well as other approaches, to a set of studies examining polybrominated diphenyl ethers (PBDEs) and neurodevelopment. They concluded that use of the IRIS approach resulted in exclusion of studies, which would lead to hazard conclusions based on an incomplete body of evidence. As scientists in the IRIS program, we support the comparison of approaches to improve systematic review methods for environmental exposures; however, we believe the IRIS approach was misrepresented. In this letter, we demonstrate that the ratings attributed to the IRIS approach were not consistent with our own application of the tool. We also clarify the use of studies rated as "low confidence" and the use of an overall study confidence rating in our systematic reviews. In conclusion, the IRIS study evaluation approach is a transparent method to inform certainty in our evidence synthesis decisions and ensures consistency in the development of IRIS health assessments.

Phthalate exposure and female reproductive and developmental outcomes: a systematic review of the human epidemiological evidence.

OBJECTIVE: We performed a systematic review of the epidemiology literature to identify the female reproductive and developmental effects associated with phthalate exposure. DATA SOURCES AND STUDY ELIGIBILITY CRITERIA: Six phthalates were included in the review: di(2-ethylhexyl) phthalate (DEHP), diisononyl phthalate (DINP), dibutyl phthalate (DBP), diisobutyl phthalate (DIBP), butyl benzyl phthalate (BBP), and diethyl phthalate (DEP). The initial literature search (of PubMed, Web of Science, and Toxline) included all studies of female reproductive and developmental effects in humans, and outcomes were selected for full systematic review based on data availability. STUDY EVALUATION AND SYNTHESIS METHODS: For each outcome, studies were evaluated using criteria defined a priori for risk of bias and sensitivity by two reviewers using a domain-based approach. Evidence was synthesized by outcome and phthalate and strength of evidence was summarized using a structured framework. RESULTS: The primary outcomes reviewed here are (number of included/excluded studies in parentheses): pubertal development (5/13), time to pregnancy (3/4), preterm birth (8/12), and spontaneous abortion (5/0). Among these outcomes, preterm birth had moderate evidence of a positive association with phthalate exposure (specifically DEHP, DBP, and DEP). Exposure levels for BBP, DIBP, and DINP were generally lower than for the phthalates with an observed effect, which may partially explain the difference due to lower sensitivity. Other phthalate/outcome combinations were considered to have slight or indeterminate evidence of an association. CONCLUSIONS AND IMPLICATIONS OF KEY FINDINGS: Overall, these results support that some phthalates may be associated with higher odds of preterm birth in humans, though there is some remaining inconsistency. More evidence is needed on the mechanism and relevant exposure window for this association. The views expressed are those of the authors and do not necessarily represent the views or policies of the U.S. EPA.

Epidemiology: a foundation of environmental decision making.

Many epidemiologic studies are designed so they can be drawn upon to provide scientific evidence for evaluating hazards of environmental exposures, conducting quantitative assessments of risk, and informing decisions designed to reduce or eliminate harmful exposures. However, experimental animal studies are often relied upon for environmental and public health policy making despite the expanding body of observational epidemiologic studies that could inform the relationship between actual, as opposed to controlled, exposures and health effects. This paper provides historical examples of how epidemiology has informed decisions at the U.S. Environmental Protection Agency, discusses some challenges with using epidemiology to inform decision making, and highlights advances in the field that may help address these challenges and further the use of epidemiologic studies moving forward.

Study sensitivity: Evaluating the ability to detect effects in systematic reviews of chemical exposures.

A critical step in systematic reviews of potential health hazards is the structured evaluation of the strengths and weaknesses of the included studies; risk of bias is a term often used to represent this process, specifically with respect to the evaluation of systematic errors that can lead to inaccurate (biased) results (i.e. focusing on internal validity). Systematic review methods developed in the clinical medicine arena have been adapted for use in evaluating environmental health hazards; this expansion raises questions about the scope of risk of bias tools and the extent to which they capture the elements that can affect the interpretation of results from environmental and occupational epidemiology studies and in vivo animal toxicology studies, (the studies typically available for assessment of risk of chemicals). One such element, described here as "sensitivity", is a measure of the ability of a study to detect a true effect or hazard. This concept is similar to the concept of the sensitivity of an assay; an insensitive study may fail to show a difference that truly exists, leading to a false conclusion of no effect. Factors relating to study sensitivity should be evaluated in a systematic manner with the same rigor as the evaluation of other elements within a risk of bias framework. We discuss the importance of this component for the interpretation of individual studies, examine approaches proposed or in use to address it, and describe how it relates to other evaluation components. The evaluation domains contained within a risk of bias tool can include, or can be modified to include, some features relating to study sensitivity; the explicit inclusion of these sensitivity criteria with the same rigor and at the same stage of study evaluation as other bias-related criteria can improve the evaluation process. In some cases, these and other features may be better addressed through a separate sensitivity domain. The combined evaluation of risk of bias and sensitivity can be used to identify the most informative studies, to evaluate the confidence of the findings from individual studies and to identify those study elements that may help to explain heterogeneity across the body of literature.

Use of health information in air pollution health research: past successes and emerging needs.

In September 2006, the US Environmental Protection Agency and the US Centers for Disease Control (CDC) co-organized a symposium on "Air Pollution Exposure and Health." The main objective of this symposium was to identify opportunities for improving the use of exposure and health information in future studies of air pollution health effects. This paper deals with the health information needs of such studies. We begin with a selected review of different types of health data and how they were used in previous epidemiologic studies of health effects of ambient particulate matter (PM). We then examine the current and emerging information needs of the environmental health community, dealing with PM and other air pollutants of health concern. We conclude that the past use of routinely collected health data proved to be essential for activities to protect public health, including the identification and evaluation of health hazards by air pollution research, setting standards for criteria pollutants, surveillance of health outcomes to identify incidence trends, and the more recent CDC environmental public health tracking program. Unfortunately, access to vital statistics records that have informed such pivotal research has recently been curtailed sharply, threatening the continuation of the type of research necessary to support future standard setting and research on emerging exposure and health problems (e.g. asthma, multiple sclerosis, diabetes, and others), as well as our ability to evaluate the efficacy of regulatory and other prevention activities. A comprehensive devoted effort, perhaps new legislation, will be needed to address the standardization, centralization, and sharing of data sets, as well as to harmonize the interpretation of confidentiality and privacy protections across jurisdictions. These actions, combined with assuring researchers and public health practitioners appropriate access to data for evaluation of environmental risks, will be essential for the achievement of our environmental health protection goals.

Changes in systolic blood pressure associated with lead in blood and bone.

BACKGROUND: Several studies have examined longitudinal associations of blood pressure change or hypertension incidence with lead concentration in blood or bone. It is not clear whether the observed associations reflect an immediate response to lead as a consequence of recent dose or rather are a persistent effect of cumulative dose over a lifetime. METHODS: We followed 575 subjects in a lead-exposed occupational cohort in South Korea between October 1997 and June 2001. We used generalized estimating equation models to evaluate blood pressure change between study visits in relation to tibia lead concentrations at each prior visit and concurrent changes in blood lead. The modeling strategy summarized the longitudinal association of blood pressure with cumulative lead dose or changes in recent lead dose. RESULTS: On average, participants were 41 years old at baseline and had worked 8.5 years in lead-exposed jobs. At baseline, the average +/- standard deviation for blood lead was 31.4 +/- 14.2 microg/dL, and for tibia lead, it was 38.4 +/- 42.9 microg/g bone mineral. Change in systolic blood pressure during the study was associated with concurrent blood lead change, with an average annual increase of 0.9 (95% confidence interval = 0.1 to 1.6) mm Hg for every 10-microg/dL increase in blood lead per year. CONCLUSION: The findings in this relatively young population of current and former lead workers suggest that systolic blood pressure responds to lead dose through acute pathways in addition to the effects of cumulative injury.

The longitudinal association of lead with blood pressure.

BACKGROUND: Several investigators have reported an association of blood lead or bone lead with increased blood pressure and hypertension, but questions remain concerning whether these effects are acute or chronic in nature. METHODS: In this longitudinal study, we evaluated the relation of lead, measured in blood and tibia, to changes in blood pressure between 1994 and 1998. We studied 496 current and former employees of a chemical-manufacturing facility in the eastern United States who had previous occupational exposure to inorganic and organic lead. Cohort members who provided three or four blood pressure measurements during the study were included. RESULTS: Mean age at baseline was 55.8 years with a mean of 18 years since last occupational exposure to lead. Blood lead at baseline averaged 4.6 microg/dL (standard deviation [SD] = 2.6) or 0.22 micromole/Liter (SD = 0.13). Tibia lead at year three averaged 14.7-microg/gm (SD = 9.4) bone mineral. Change in systolic blood pressure during the study was associated with lead dose, with an average annual increase of 0.64 mmHg (standard error [SE] = 0.25), 0.73 mmHg (SE = 0.26), and 0.61 mmHg (SE = 0.27) for every standard deviation increase in blood lead at baseline, tibia lead at year three, or peak past tibia lead, respectively. CONCLUSIONS: The results support an etiologic role for lead in the elevation of systolic blood pressure among adult males and are consistent with both acute and chronic modes of action.